Optimal Step Size for the Adaptive Least-Mean Squares Algorithm Applied in the Fractional Fourier Transform Domain for Efficient Signal Estimation in Interference and Noise

The Fractional Fourier Transform (FrFT) has wide applications in communications and signal processing. It has been shown to provide significant improvement over the conventional Fourier Transform when the signal-of-interest (SOI) or the interference and noise environment is nonstationary, as is often the case. Recently, a Least-Mean Squares (LMS) algorithm was developed for computing the optimum weight vector of the adaptive linear filter applied in the FrFT domain to estimate a SOI from the environment. A computational complexity exists, however, in that typically the FrFT rotational parameter ‘a’ cannot be mathematically computed a priori, and with the LMS algorithm, there is also a step size μ that must be computed. Finding the optimum rotational parameter and optimum step size could require several iterations to achieve the minimum mean-square error (MMSE) for this adaptive filtering method. Since the rotational parameter cannot in general be calculated mathematically, in this paper we develop a method to determine the optimum step size μopt mathematically, thereby improving practical use of the algorithm. We show that the optimum step size produces the MMSE, as desired, and in fact, the step size itself converges to the MMSE as a function of time. We demonstrate the proposed method by simulation of a BPSK signal in non-stationary noise. Keywords—Adaptive Filtering, Fractional Fourier Transform, Least-Mean Squares, Minimum Mean-Square Error, Step Size.